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Abstract 

In response to changing traffic and staffing condi- 
tions, supervisors dynamically configure airspace 
sectors by assigning them to control positions. A 
finite horizon airspace sector configuration prob- 
lem models this supervisor decision. The prob- 
lem is to select an airspace configuration at each 
time step while considering a workload cost, a re- 
configuration cost, and a constraint on the num- 
ber of control positions at each time step. Three 
algorithms for this problem are proposed and 
evaluated: a myopic heuristic, an exact dynamic 
programming algorithm, and a rollouts approxi- 
mate dynamic programming algorithm. On prob- 
lem instances from current operations with only 
dozens of possible configurations, an exact dy- 
namic programming solution gives the optimal 
cost value. The rollouts algorithm achieves costs 
within 2% of optimal for these instances, on av- 
erage. For larger problem instances that are rep- 
resentative of future operations and have thou- 
sands of possible configurations, excessive com- 
putation time prohibits the use of exact dynamic 
programming. On such problem instances, the 
rollouts algorithm reduces the cost achieved by 
the heuristic by more than 15% on average with 
an acceptable computation time. 

Nomenclature 

a Parameter in workload cost 

8w(X) 

A Adjacency matrix for sectors 


w(c,T) 

Workload threshold in control 
position c when the traffic situ- 
ation is T 

w(X) 

A M x 1 vector in which the z' th 
component is w(ci,T), Ci G C 

P 

Parameter in single time step 
cost g{X k ,u k ,X k+ \ ) 

f 

Uncertain prediction of future 
traffic 

w(c,T) 

Expected workload level in con- 
trol position c when the traffic 
prediction is t 

Hot) 

Expected workload threshold in 
control position c when the traf- 
fic prediction is t 

c 

Set of all valid configurations 

c, 

Set of all valid configurations 
with i control positions 

Mieur (A , d ) 

Heuristic control policy; out- 
puts a control action 

FkiXjc) 

Indicates what control action to 
take at time step k and state X k 

Tt 

Control policy specifying p k for 
all time steps of interest 

J~k(X k ) 

Rollouts approximate optimal 
cost-to-go from state X k at time 
step k 

w(X) 

A M x 1 vector in which the z th 
component is w(ci,T), ct G C 

c 

A configuration; a set of control 
positions 

c 

A control position; a set of sec- 
tors 

d k 

Required number of control po- 
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g(^b M G^fc+l) 

sitions at time step k 
Single time step cost 

Sri^ki 1) 

Reconfiguration cost 

g w (X) 

Workload cost 

H 

Number of Monte Carlo simula- 

W 

tions 

Optimal cost-to-go from state 

K 

Xk at time step k 
Final time step 

k 

Time step 

L 

Reduced number of time steps; 

M 

L< K 

The number of control positions 

N 

Number of sectors 

S 

A set of sectors to be configured 

s 

A sector 

T 

State and flight plan informa- 

u k 

tion for all aircraft that are cur- 
rently in or are planning to fly 
through the sectors of interest 
Control action time step k: 

u k 

equal to Q + i 

Optimal action at time step k 

w(cj) 

Workload in control position c 

X 

when the traffic situation is T 
System state; includes C and T 


1 Introduction 

In current air traffic management operations, 
airspace is partitioned into static volumes called 
sectors to facilitate the division of responsibili- 
ties between air traffic controller teams. A sec- 
tor configuration maps a set of airspace sectors 
into a set of control positions such that each sec- 
tor is assigned to exactly one control position. A 
single team of one to three air traffic controllers 
monitors each control position. Supervisors se- 
lect configurations so as to 1) assign enough con- 
troller teams to busy airspace and 2) avoid work- 
load associated with transitions between config- 
urations [20]. Current air traffic control oper- 
ations require so much memorization that each 
controller is only certified to control a set of six 
to eight sectors referred to as an “area of special- 
ization," such as the area shown in figure 1. The 
relatively small number of sectors in each area 


mean that there are a relatively small number of 
permitted sector configurations in each area. 



Fig. 1 The six sectors in Area 7 of Cleveland Air 
Route Traffic Control Center. 

With planned improvements in automa- 
tion and changes in operations proposed by 
MITRE [1,28], Mogford [31-33], and the Fed- 
eral Aviation Administration [11], controllers 
will be able to control a wider range of sectors. 
This will increase the number of possible config- 
urations. This increase in configuration options 
makes it more likely that an appropriate sequence 
of configurations will exist to fit any given traf- 
fic and controller resource situation. However, it 
will also make the selection of appropriate sec- 
tor configurations more challenging for supervi- 
sors. Therefore, a tactical decision- support tool 
that suggests good sector configurations will aid 
supervisors in this context [11]. 

Several algorithms for configuring sectors 
that could be used in such a decision- support 
tool have been proposed [4, 5, 7, 9, 10, 12-19,36]. 
These algorithms vary in the version of the sector 
configuration problem they attempt to solve. One 
supports tactical decision-making [9], but the rest 
focus on pre-tactical or strategic decisions con- 
cerned with staff planning. Some attempt to min- 
imize the number of control positions required to 
safely manage the traffic [4, 5, 10]. Others set 
a required number of control positions at each 
time [7] or put an upper bound on the number 
of control positions at each time [9, 16, 17, 36]. 
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These algorithms also vary in how they quan- 
tify controller workload. Workload is quantified 
using aircraft count [4,5, 10], inter-control posi- 
tion coordination [7], control position through- 
put [36], detailed human task load models [9], or 
neural networks trained on historical traffic data 
and sector configurations [13,15,18,19]. Further- 
more, none of the existing algorithms explicitly 
consider the uncertainties involved in predictions 
of air traffic. Such uncertainties impact predic- 
tions of any measure of controller workload. 

The workload associated with reconfigura- 
tions is treated differently by existing algorithms 
for configuring sectors. Several algorithms ig- 
nore this reconfiguration workload [4,5,9, 10], 
but some take steps to control the frequency of 
reconfigurations [12, 14, 15,36]. None specify 
an explicit reconfiguration cost that considers the 
differing levels of workload induced by different 
reconfigurations, even though such differences 
exist and have been studied [20,21,26,27]. 

These algorithms also vary in the solution 
methods they utilize to select configurations. The 
methods include original heuristics [4,5], integer 
programming [10,36], genetic algorithms [7, 16, 
17], brute-force exhaustive search [9], and tree- 
search methods like branch-and-bound and the 
A* algorithm [12, 14-17]. 

In this paper, a new formulation of the tactical 
sector configuration problem is proposed. This 
formulation explicitly considers uncertainties in 
predictions of air traffic by using predictions of 
future air traffic sampled from an appropriate dis- 
tribution. The control actions in this problem are 
the sector configurations at each time step over 
a finite time horizon. These configurations must 
contain a required number of control positions at 
each time step. The objective is to minimize the 
expected value of a weighted sum of a workload 
cost and a reconfiguration cost. Three algorithms 
for this problem are proposed and evaluated. One 
of these algorithms is a myopic heuristic, another 
is an exact dynamic programming algorithm that 
can be solved with value iteration, and the third 
uses rollouts to generate an approximate dynamic 
programming solution. 

The rest of this paper is structured as follows. 


The new sector configuration problem is speci- 
fied in Section 2. The algorithms for this problem 
are described in Section 3. In Section 4, the per- 
formance of the algorithms is analyzed by apply- 
ing them to small problem instances from current 
operations and large problem instances from pos- 
sible future operations. Section 5 contains con- 
clusions and Section 6 contains suggestions for 
future work. 

2 Problem Specification 

The problem is to select an airspace configuration 
at each time step while considering a workload 
cost, a reconfiguration cost, and a constraint on 
the number of control positions at each time step. 
The problem specification consists of a system 
model and dynamics, problem constraints, and an 
objective function. 

Let S = { 51 , 52 , • • • , $n} be the set of N sectors 
to be configured. The matrix A is an adjacency 
matrix that specifies which sectors are neighbors. 
Element A (/ is 1 if s, is adjacent to sj, other- 
wise it is 0. A configuration of S is specified by 
C = {ci,C 2 , . • • ,cm}, where M is the number of 
control positions. Each control position c,- e C is 
itself a set that contains one or more sectors. 

2.1 System Model 

The system state at time step k is X k = (Q, T k ), 
where C k is the sector configuration at k and T k 
specifies the aircraft traffic situation for the sec- 
tors in S at k. T k contains state and flight plan 
information for all aircraft that are currently in or 
are planning to fly through the sectors in S. It 
contains all the aircraft data available for predict- 
ing future traffic situations. 

The size of the state space for C k can be in 
the thousands or millions for problems of in- 
terest in the future air traffic management sys- 
tem. For one such problem studied in Section 
4, there are 8,362 valid configurations of 11 sec- 
tors. For one current-day problem in the French 
airspace, there are around 124,000 valid config- 
urations [17]. The size of the state space for T k 
is infinite because aircraft state variables can be 
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continuous. 

The control at a time step sets the configura- 
tion for the next time step: Uk — Q+i. Aircraft 
trajectory predictors can be used to predict the 
traffic situation at future time steps, but there is 
no simple function or stationary statistical distri- 
bution that specifies the dynamics of T^. How- 
ever, there are some distributions available that 
describe the dynamics of the number of aircraft in 
a sector [37]. Rather than explicitly modeling the 
dynamics of traffic, these distributions are used 
to model the dynamics of the number of aircraft 
in each sector, one measure of workload, as will 
be discussed in the next two sections. 

2.1.1 Workload 

There are many functions that attempt to quan- 
tify the workload in a control position, given the 
control position and the current traffic [6, 22-25, 
29,35]. Let the selected workload function be 
specified by w{c,T). Let w(X) return a |Cj x 1 
vector in which the z th component is the work- 
load for Ci € C and where X — (C. T). Uncertain 
predictions of future traffic T can also be passed 
into the workload function, yielding correspond- 
ing predictions of the control position workloads. 

There is a workload threshold associated with 
each control position. This threshold is not a 
hard constraint, but as workload approaches and 
exceeds the threshold, operations in the control 
position probably become less safe and less effi- 
cient. This threshold may depend on character- 
istics of the traffic in the sector, such as sector 
transit times [38]. Let w(c,T) specify the work- 
load threshold for c when the traffic situation is 
T, and let w(X) be a |Cj x 1 vector in which 
the z th component is the workload threshold for 
Ci e C. Generally, the workload threshold for a 
control position is much smaller than the sum of 
the thresholds of the individual sectors that make 
up the control position. 

2.1.2 Workload Dynamics 

Wanke et al. used historical prediction errors to 
estimate distributions that describe the dynam- 


ics of a measure of workload [37]. This work- 
load measure, which is used in the current air 
traffic management system, is the peak aircraft 
count that occurs in a sector during a 15 -minute 
time period. Wanke et al. specify a distribution 
for future peak aircraft count based on predic- 
tion lookahead time, sector type, and properties 
of a particular prediction of the future traffic in 
the sector (peak predicted number of aircraft in 
the sector during the time period and the fraction 
of those aircraft that have departed at the time of 
the prediction). Unfortunately, there are no pub- 
lished joint or conditional distributions capturing 
the relationship between peak aircraft count lev- 
els over time and in different control positions. 

For this research, the 15-minute lookahead 
distributions of peak aircraft count derived by 
Wanke et al. are used to model the dynamics of 
the workload in each sector [37]. Algorithms also 
use these distributions when making predictions 
about the future, even if the predictions are made 
with a lookahead time that is larger than 15 min- 
utes. It is assumed that the distributions are in- 
dependent across sectors and time steps because 
no joint or conditional distributions are available. 
The traffic-related distribution parameters (pre- 
dicted number of aircraft in the sector and the 
fraction of those aircraft that have departed at the 
time of the prediction) are computed from his- 
torical traffic rather than predicted traffic because 
the operational prediction data is not available. A 
sample distribution is shown in figure 2. 

This approach allows for algorithm perfor- 
mance evaluations based not only on the single 
sample path of historical traffic but also on other 
sample paths that could have occurred. This sort 
of evaluation is essential when evaluating any al- 
gorithm that claims to be robust to uncertainties. 
Predictions that are consistent with the workload 
dynamics can be generated by sampling the in- 
dependent distributions that govern the system 
dynamics, enabling algorithms to accurately and 
quickly approximate expected values of future 
costs. This approach assumes that samples in 
nearby sectors and at nearby times are indepen- 
dent. A second assumption is that uncertainties 
in predictions are constant with respect to looka- 
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Fig. 2 Probability mass function for the 12:45 
pm (15-minute lookahead) prediction of the max- 
imum number of aircraft in sector ZOB38 in 
Cleveland Center between 1:00 pm and 1:15 pm 
local time on 22 February 2007. 


head time. The impact of these assumptions and 
alternative approaches to handling system uncer- 
tainties should be evaluated in future research. 

2.2 Objective 

There are two components of the cost at each 
stage: a workload cost g w and a reconfiguration 
cost g r . The total cost g is a weighted sum of the 
two. 

The workload cost penalizes cases where the 
w(-X)t) exceeds w(X^). This cost function starts 
with the amount by which the workload exceeds 
the workload threshold in each control position, 
multiplies it by the parameter a, squares the re- 
sult, and finally sums the results for all the control 
positions in Q: 

g w (X k ) = ||a[w(X fc ) -w(X*)]+ll2> (!) 

where the operators are defined as follows. For 
a real input a, [a]+ is 0 if a < 0 and a if a > 0. 
When operating on a vector, as in (1), the opera- 
tor [•] + applies separately to each element in the 
vector. Also, for a vector a, ||«||^ is the squared 
2-norm of a. Control positions with workload 
levels below the threshold are not penalized. Al- 
ternative workload cost function formulations are 


possible. Airspace sector design and configura- 
tion research has pursued objectives like work- 
load balance, traffic flow conformance, and low 
inter-control position coordination [8, 30, 34, 39] ; 
these could be added to this cost function in fu- 
ture work. 

The next component is the reconfiguration 
cost, which increases as the next configuration 
becomes more different from the current con- 
figuration. Significant configuration changes in- 
duce workload for air traffic control staff as 
they transition from one control position to an- 
other [20,21,26,27]. This cost specifies the num- 
ber of control positions in Q+i that were not 
present in Q: 

gr(Xk,Uk,X k+ 1) = |Q+1 \Q|, (2) 

where \ is the set minus operator. Al- 
ternate reconfiguration cost formulations 
may be considered. Gianazza et al. use 
min{|Q|,|Q +i |} - \C k n C k+i \ as a distance 
function in a genetic algorithm [17]. The 
authors of this paper have previously used 
| Q \ Q. |-i | [20]. These possibilities are inde- 
pendent of the traffic situations T k and T k+ \ , but 
the reconfiguration workload does depend on 
the traffic situation, and more sophisticated cost 
functions may consider the traffic in control po- 
sitions that are being combined and split to move 
from one configuration to the next [21,26,27]. 

The overall single time step cost is simply the 
weighted sum of equations (1) and (2): 


§{Xk, H-ki^k+l) — 8w(Xk) A 

(3) 

It is parameterized by a and [1 a nonnegative 
weight on the reconfiguration cost. Appropriate 
values for a and p are a topic for future research. 

The overall problem considered here is a fi- 
nite horizon expected total cost problem with the 
objective 


min E 


K - 1 

^ g(Xk,Uk,X k+ i) 
k = 0 


*0 


, (4) 


subject to the system model described in sub- 
section 2.1 and the constraints discussed in sub- 
section 2.3. For some random variable a and a 
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variable it depends on b, E[a|fr] denotes the ex- 
pected value of a given b. Also, n is a control 
policy in which each /u k (X k ) indicates what con- 
trol action to take at time step k and state X k . 

2.3 Constraints 

A valid C must assign each .?,• G S to a control 
position: 

M 

Ug = S, (5) 

j=l 

where U is the set union operator. A valid C also 
cannot assign any Sj to more than one control po- 
sition: 

a n Cj = 0 V ci-cj ec.i^ j. (6) 

where D is the set intersection operator and 0 is 
the empty set. These two requirements ensure 
that 1 < M < N. All the S( in a valid c must define 
a single connected component in the graph spec- 
ified by A. This ensures that sectors in the same 
control position are adjacent to each other. 

Furthermore, certain control positions may 
not be allowed for reasons such as a preference 
for convexity in the top-view of control positions 
or a maximum control position size constraint. 
Any valid configuration C contains no disallowed 
control position. 

The set C contains all valid configurations of 
a given set of sectors S. Furthermore, let C, be 
the set of all valid configurations with i control 
positions (i.e. |Cj = i V C G C,). It is easy to 
see that |Ci | = 1 and |Qv| = 1, but there may be 
many valid configurations when the number of 
control positions is between 1 and N. There is a 
constraint on the control that specifies the num- 
ber of control positions that can be used at each 
time step. This constraint is needed because a 
certain number of controller teams are available 
and should be working at each time step. Let d k 
specify the required number of control positions 
at k. Therefore, u k G C d k+1 is a constraint. 

3 Algorithms 

This section proposes three algorithms that at- 
tempt to solve the problem defined in Section 2. 


The first is a myopic heuristic that is similar to 
the heuristic in Refs. [4,5]. It computes a solution 
quickly but makes no guarantees on its optimal- 
ity. The next algorithm uses dynamic program- 
ming and solves for the optimal costs-to-go with 
value iteration. It computes the optimal solution 
but is computationally intensive and can be slow 
when | C | is large. The final algorithm utilizes 
an approximate dynamic programming technique 
known as rollouts. It uses the myopic heuris- 
tic to approximate the optimal costs-to-go and 
is therefore a hybrid of the first two algorithms. 
It can guarantee better performance than the my- 
opic heuristic and produces a solution faster than 
exact dynamic programming. 

3.1 Myopic Heuristic 

A myopic heuristic algorithm was designed for 
this problem. This algorithm is myopic be- 
cause when selecting u k it only considers the ex- 
pected workload levels in the next time step and 
the required number of control positions in the 
next time step. The expected workload levels 
w{cj k+l ) and workload thresholds w(c,t k+ i) 
are approximated with the mean workload from 
Monte Carlo simulations of the next time step. 

If |Q t+1 | > |QJ or w(c,f k+ i) > 

w(c,f k+i ) for some c G Q, then c max = 
argmax ceC vi>(c, t k+ ] ) is split into two control 
positions. If there are multiple ways to split 
Cmax, the split that leaves the lowest maximum 
w(c,f*+i) in the two resulting control positions 
is selected. 

If|C 4+ ,| < |C d k \, then the algorithm selects 
a combination to implement. It searches for an 
acceptable combination by starting with c m in = 
argmin c£ Q w>(c, T k+ \). All possible ways to com- 
bine c m i n with a neighboring c are investigated, 
and the combination that results in the control 
position with the lowest is implemented, assum- 
ing that expected workload is below a threshold. 
If no acceptable combination involving c m i n is 
found, the control position with the next lowest 
expected workload is considered until eventually 
an acceptable combination is determined. Some 
additional logic helps ensure that some combina- 
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tion is found so that the constraint e C^ +1 is 
not violated. 


3.2 Dynamic Programming Solution via 
Value Iteration 

A fundamental quantity in dynamic program- 
ming is the optimal cost-to-go; it is the cost that 
would be incurred if optimal actions are taken 
from some state onward. The optimal cost-to-go 
function here depends on the time step k and is 
specified as 


Jt( x k) = min E 

l 


■£ g(Xj,uj, x j +l ) 
j=k 


X k 


(7) 

This optimal cost-to-go is approximated with 
the average cost from Monte Carlo simulations 
constructed from the workload distributions dis- 
cussed in sub-section 2.1.2. The expression for 
this approximation is 


Jt(Xi) 


1 

: min — 

ftv-w- 1 H 


H 

E 

h= 1 


rh 


Carlo simulation h, h = 1,2, . . . ,H. In this finite 
time horizon case, the J^{Xj^) values can be com- 
puted with value iteration starting at the final time 
step [2]. 

The optimal action to take at each time step 
can then be found with 


incurred if a heuristic were used for the remain- 
der of the time horizon. This approach is closely 
related to policy iteration, a technique that can 
be used to solve for the optimal costs-to-go [2]. 
Rollouts is guaranteed to lead to better cost re- 
sults than are achieved by the raw heuristic that 
it uses. The approximations of the optimal cost- 
to-go are computed as needed during algorithm 
execution, which is appropriate for this problem 
because the state space is infinite. These approx- 
imations can make use of the latest traffic situa- 
tion data, so this approach will be even more ap- 
propriate for more sophisticated versions of the 
sector configuration problem in which new traf- 
fic information enables more accurate predictions 
of future traffic and therefore future costs. 

More precisely, let /Jheur(^b<4+t) be a 
heuristic control policy such as the myopic 
heuristic suggested in sub-section 3.1. The opti- 
mal cost-to-go is approximated by assuming that 
this heuristic policy is used for the remainder of 


~K— 1 

the time horizon: 


E s(x;,uj,xf + ,) 

5 

~K - 1 

-| 

J= k 

r k (x k )*E 

T. 8 (Xj , fJheui (Xj , dj+ 1 ) , Xj- 1_ i ) 

X k 


8) 

J=k 


time step j in Monte 


(10) 


As with the dynamic programming solution via 
value iteration, the expected value is approxi- 
mated with the average of H Monte Carlo sim- 
ulations. The rollouts algorithm computes its ap- 
proximate optimal cost-to-go according to 


u% e argmin E [g(X k ,u k ,X k+l )+j£ +l (X k+1 )\X k ] . 
Uk€C d k+ 1 

(9) 

Again, the expected value here is approximated 
with Monte Carlo simulations. 

3.3 Rollouts Approximate Dynamic Pro- 
gramming Solution 

Rollouts is a technique for approximating the op- 
timal cost-to-go [2]. The approximate optimal 
cost-to-go is then used in equation (9) to select 
an action at each time step. The idea is to ap- 
proximate the optimal cost-to-go at each possi- 
ble next state with the cost-to-go that would be 


Mx , 0 



Z8{Xj,to*,(Xl,dj+i),Xj+i) 


J= k 


(ID 


Three techniques are used to speed up the 
computation of the approximate optimal cost-to- 
go in equation (11). The first is to compute the 
cost-to-go incurred by the heuristic for some L 
time steps, rather than all the way until the end 
of the problem time horizon. This will hinder 
the performance of the algorithm because there 
may be a reason to select a control action that is 
not revealed until beyond the next L time steps. 
The second technique is to stop computing the 
approximate optimal cost-to-go for a configura- 
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tion when the approximation already exceeds the 
smallest approximate optimal cost-to-go found so 
far. This technique will not adversely affect the 
algorithm’s cost performance. The third method 
for speeding up computation is to reduce the 
number of Monte Carlo simulations used to ap- 
proximate the expected value. Such Monte Carlo 
simulations are computationally cheap to pro- 
duce using the workload distributions assumed 
for each sector at each time step (see sub-section 
2.1.2), so this method is probably not helpful 
for this version of the sector configuration prob- 
lem. However, some Monte Carlo simulations 
and workload metrics are computationally expen- 
sive to produce, so this method could become im- 
portant in future work. 

4 Quantitative Analysis of the Algorithms 

4.1 Small Problem Instances from Current 
Operations 

Areas of specialization typically contain six to 
eight sectors, and they make convenient small 
problem instances for studying the performance 
of the three proposed algorithms. They are also 
interesting problem instances to consider because 
historical sector configuration data is available, 
so the performance of the algorithm-generated 
configurations can be compared with the perfor- 
mance of the historical configurations. Finally, 
differences between the historical configurations 
and those selected by the algorithms can help re- 
veal problems with the algorithms [20]. A de- 
tailed comparison of this type is an opportunity 
for future research. 

For these experiments, historical air traffic 
data from three Thursdays in February of 2007 
and the eight areas of specialization in Cleveland 
Air Route Traffic Control Center were used to 
construct 19 different problem instances. Fig- 
ure 1 shows the sectors in one such area. Each 
of the eight areas considered for these problems 
contains six or seven sectors. The valid config- 
urations for each day were limited to those that 
use control positions from the historical sector 
configurations on that day. Also, the number of 


required control positions at each time step was 
set to the number in the historical data. There 
were between 12 and 24 possible configurations 
for each problem instance. The number of con- 
figurations with the required number of control 
positions at each time step ranged between one 
and nine but averaged only about three. 

Each problem instance ran from 3:45 am to 
10:45 pm local time. The time steps were 15 
minutes long, but the algorithms did not usually 
change airspace configurations every 15 minutes 
due to the reconfiguration cost. For each of the 
19 instances, 10,000 Monte Carlo runs were cre- 
ated to help determine how each algorithm would 
do under a variety of possible workload sam- 
ple paths based on the recorded traffic sample 
path. These runs were generated using the in- 
dependent distributions discussed in sub-section 
2. 1 .2. The dynamic programming algorithm used 
a different set of 10,000 Monte Carlo simulations 
to compute expected values, while the heuristic 
and rollouts algorithms used only 100 such sim- 
ulations in an attempt to increase their compu- 
tational speed. The rollouts algorithm approx- 
imated the optimal cost-to-go by running the 
heuristic for four hours rather than for the entire 
remainder of the problem. Finally, workload was 
specified as the maximum number of aircraft in a 
control position during the time step divided by a 
control position-specific upper bound called the 
Monitor Alert Parameter (MAP) so that all work- 
loads ranged between zero and about one. For 
some control positions, published MAP values 
are available in the Future ATM Concepts Eval- 
uation Tool [3]. If no published value was avail- 
able, the MAP value was set to the largest of the 
MAP values of the sectors in the control position. 
The workload threshold returned by w (c, T) was 
set to 0.9 for all control positions and traffic sit- 
uations. In the single stage cost function, the a 
parameter was set to 10 and the [1 parameter was 
set to 1. This means that the workload cost of 
the workload equaling the threshold in a control 
position for one time step is equal to the recon- 
figuration cost of one new control position. Table 
1 contains a summary of the parameters used for 
these analyses. 
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Table 1 Problem and Algorithm Parameters for Small Operational Problem Instances 


Parameter 

Value 

Problem start time 

3:45 am Eastern Time 

Problem end time 

10:45 pm Eastern Time 

Time step duration 

15 minutes 

K 

75 

L 

16 

H for dynamic programming 

10,000 

H for heuristic and rollouts 

100 

w(cj) 

maximum aircraft count during time step 
monitor alert parameter 

w(c,T) 

0.9 

a 

10 

P 

1 


Table 2 Cost and Computation Time Results for Small Operational Problem Instances 


Approach 

Fraction of Optimal Cost (g) 
Min Mean Max 

Mean Compute Time 
per Time Step [seconds] 

Dynamic Programming 

1.000 

1.000 

1.000 

0.095 

Historical 

1.000 

1.217 

2.174 

— 

Heuristic 

1.000 

1.165 

2.100 

0.012 

Rollouts 

1.000 

1.014 

1.159 

0.065 


Table 2 summarizes the cost and computation 
time results for the Monte Carlo runs of these 19 
problem instances. The dynamic programming 
approach computes the optimal solution to the 
problems, assuming that it uses enough Monte 
Carlo simulations to accurately approximate ex- 
pected values. The costs incurred by the other 
approaches are expressed as a fraction of the op- 
timal cost found by dynamic programming. Fig- 
ure 3 graphically portrays the information in Ta- 
ble 2. Note that the compute time per time step 
is set to zero for the operational configurations 
because the actual compute time in operations is 
not known. The historical sector configurations 
perform relatively well on these small problems. 
On at least one problem instance they achieve the 
optimal cost, and on average the cost from the 
historical configurations is within about 22% of 
optimal. However, on one problem instance the 
historical sector configurations led to a cost that 
was more than double the optimal cost. The per- 


formance of the heuristic is similar to that of the 
historical configurations, but slightly better. The 
rollouts algorithm achieves a cost that is within 
2% of optimal on average and never more than 
16% from optimal on these 19 problem instances. 

The dynamic programming algorithm took 
about a tenth of a second per time step to select 
a configuration. It benefitted from the fact that 
all of the workload distributions were assumed 
to be fixed and independent. This meant that the 
dynamic programming algorithm only had to per- 
form value iteration at the first time step, and then 
could re-use the optimal costs-to-go at all future 
time steps. In a more realistic problem formu- 
lation where new information becomes available 
over time and changes the distribution of future 
traffic and workload, the dynamic programming 
solution would have to perform value iteration at 
each time step, and it would therefore be slower. 
The heuristic algorithm computed a solution in 
about a hundredth of a second. The rollouts algo- 
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Mean Compute Time per Time Step [seconds] 

Fig. 3 Mean compute time per time step and 
spread of cost results for each algorithm. The 
bottom of the vertical line is the lowest cost re- 
sult, the bottom of the box is the 25 th percentile, 
the horizontal line is the median, the top of the 
box is the 75 th percentile, and the top of the ver- 
tical line is the maximum. 

rithm took about two-thirds of the time required 
by the dynamic programming solution per time 
step to select a configuration. The efforts to speed 
up the rollouts algorithm discussed in sub-section 
3.3 approximately halved its computation time. 

4.2 Large Problem Instances from Possible 

Future Operations 

A hypothetical future area of specialization con- 
sistent with the research of Mogford [31-33] was 
constructed by slightly modifying the super-high 
altitude sectors in Cleveland Air Route Traffic 
Control Center. It contains 11 sectors, all with 
altitude floors at 34,000 feet, and is shown in fig- 
ure 4. 

Three problem instances from the same three 
Thursdays in February of 2007 were created for 
this possible area. For these instances, there are 
8,362 possible valid configurations. At each time 
step, the number of valid control actions ranged 
between 1 and 2,288 and averaged almost 500. 
The required number of control positions for each 
problem instance was set so that the total number 
of aircraft in the area divided by the number of 
control positions would not be larger than 10 air- 



Fig.4 The 1 1 sectors in a possible future area of 
Cleveland Air Route Traffic Control Center. 

craft. The other details relating to these instances 
are the same as for the smaller problem instances 
as described in sub-section 4.1, except that these 
simulations ran for one more time step. 

An exact dynamic programming solution was 
attempted for one of these problem instances but 
after 10 hours it had still not finished comput- 
ing the optimal costs-to-go. Since it takes pro- 
hibitively long to compute the optimal solution 
with exact dynamic programming problem in- 
stances of this size, the cost results in table 3 are 
expressed in terms of a percentage improvement 
over the cost achieved by the heuristic. The roll- 
outs approximate dynamic programming solution 
achieves a cost reduction of between 10% and 
28% for these three problem instances, with an 
average cost reduction of more than 15%. 

Information and automation tools are being 
developed to enable larger areas of specializa- 
tion [1, 11, 28, 31-33]. If the heuristic performs 
better than configurations selected by a human 
supervisor in larger areas of specialization, as it 
does in current areas of specialization, then the 
rollouts solution would offer a cost improvement 
of 15% or more over configurations selected by 
a human supervisor in larger areas of specializa- 
tion. This should be investigated in future re- 
search, as should the benefits over current opera- 
tions of larger areas of specialization configured 
with the algorithms proposed here. 


Operational 
- Heuristic 

Dynamic Programming 
Rollouts 



0 0.02 0.04 0.06 0.08 0.1 
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Table 3 Cost and Computation Time Results for Large Possible Future Problem Instances 


Approach 

Percent Cost (g) Improvement 
over Heuristic 

Min Mean Max 

Mean Compute Time 
per Time Step 
[seconds] 

Heuristic 

Rollouts 

10.83% 17.87% 27.50% 

0.021 

97.0 


The heuristic algorithm computes a solution 
for each time step in about two hundredths of 
a second on average, which is about twice as 
long as it took for the smaller problems in sub- 
section 4.1. The rollouts algorithm takes on av- 
erage 100 seconds to compute each next config- 
uration. This should be acceptable because this 
computation would only need to complete within 
the 15 -minute time step. 

5 Conclusions 

A finite horizon airspace sector configuration 
problem was formulated. It considers workload 
costs, reconfiguration costs, and a constraint on 
the number of control positions at each time step. 
Three algorithms for this problem were proposed 
and evaluated. On small problem instances from 
current operations, an exact dynamic program- 
ming solution can be computed with value it- 
eration. This solution gives the optimal cost 
value for the problem instances. Historical sector 
configurations and those generated by a myopic 
heuristic on average led to costs that were about 
22% and 17% larger than the optimal cost, re- 
spectively. A rollouts approximate dynamic pro- 
gramming algorithm based on the myopic heuris- 
tic achieved costs within 2% of optimal, on aver- 
age. For large problem instances based on pos- 
sible future operations, the rollouts algorithm re- 
duced the cost achieved by the heuristic by more 
than 15% on average with an acceptable compu- 
tation time. 

6 Future Work 

There are several opportunities for extensions 
of this work. A more sophisticated measure 


of workload than aircraft count divided by con- 
trol position monitor alert parameter should be 
used. A more detailed comparison of histori- 
cal and algorithmically-generated configurations 
may reveal ways to improve the algorithms and 
their cost function, particularly with the help of 
airspace operations experts. The reconfigura- 
tion cost in particular may need further research 
and refinement because research on reconfigura- 
tion costs is still in its early stages [21, 26, 27]. 
Furthermore, traffic and workload uncertainties 
should be modeled more realistically. Finally, 
the algorithms proposed here should be used to 
approximate the benefits of increasing the size of 
areas of specialization and thereby allowing more 
possible configurations. 
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